Process-oriented metrics to provide feedback and assess the performance of students who are learning surgical procedures: The percutaneous dilatational tracheostomy case.

PURPOSE: Assessing competency in surgical procedures is key for instructors to distinguish whether a resident is qualified to perform them on patients. Currently, assessment techniques do not always focus on providing feedback about the order in which the activities need to be performed. In this research, using a Process Mining approach, process-oriented metrics are proposed to assess the training of residents in a Percutaneous Dilatational Tracheostomy (PDT) simulator, identifying the critical points in the execution of the surgical process. MATERIALS AND METHODS: A reference process model of the procedure was defined, and video recordings of student training sessions in the PDT simulator were collected and tagged to generate event logs. Three process-oriented metrics were proposed to assess the performance of the residents in training. RESULTS: Although the students were proficient in classic metrics, they did not reach the optimum in process-oriented metrics. Only in 25% of the stages the optimum was achieved in the last session. In these stages, the four more challenging activities were also identified, which account for 32% of the process-oriented metrics errors. CONCLUSIONS: Process-oriented metrics offer a new perspective on surgical procedures performance, providing a more granular perspective, which enables a more specific and actionable feedback for both students and instructors.

Development of a comprehensive Percutaneous Dilatational Tracheostomy process model for procedural training: A Delphi-based experts consensus.

BACKGROUND: Deconstructing a complex procedure improves skills learning, but no model has covered all relevant Percutaneous Dilatational Tracheostomy (PDT) procedural aspects. Moreover, the heterogeneity of techniques described may hinder trainees' competency acquisition. Our objective was to develop a PDT model for procedural training that includes a comprehensive step-by-step design. METHODS: Procedural descriptions were retrieved after a structured search in medical databases. Activities were extracted and the adherence to McKinley's dimensions of procedural competence was analyzed. We developed a comprehensive PDT model, which was further validated through a Delphi-based consensus of Spanish-speaking international experts. RESULTS: The 14 descriptions retrieved for analysis presented a median [interquartile range] of 18 [11-22] steps, covering 3 [2-4] of McKinley's dimensions. The Delphi panel's first model included all McKinley's dimensions, and was answered by 25 experts from nine countries, ending in the second round. The final model included 59 activities divided into six stages (51 from the initial model and eight proposed by experts) and performed by two operators (bronchoscopy and tracheostomy). CONCLUSIONS: We have presented a PDT model that includes necessary competence dimensions to be considered complete. The model was validated by an experts' consensus, allowing to improve procedural training to promote safer patient care.